Welcome to DataPipe!

For new users

Remember - just press F1 for help at any time, or choose Index or Contents from the Help Menu.
  • What is DataPipe?
  • What can I use DataPipe for?
  • How do I buy DataPipe?
  • Frequently Asked Questions and demonstrations (online)

For previous users

  • What's new in this version?

Updates

We're always adding enhancements to DataPipe. If you'd like to be notified when enhancements are available, please subscribe to the DataPipe update list below.

DataPipe Update List
This is a newsletter that is notified of all updates and changes to DataPipe. Only Crystal Software can post to this list.

See also


Features

DataPipe leverages the amazing text processing abilities of TextPipe Pro with live data in your production database. DataPipe scans through your dataset row by row (minimising locking problems), enabling full production access without interruption. Best of all, you don't have to extract or dump your data to a file before operating on it - DataPipe connects to your database live!

All the best text processing features of TextPipe Pro are here:

  • Reknowned search and replace with pattern matching, sounds-like and fuzzy matching (for fixing up common typing errors)
  • VBScript or JScript (or PerlScript, PythonScript, REXXscript etc) filters can define custom processing for each field
  • Convert end-of-line characters between Unix, Mainframe, DOS and Macintosh formats
  • Correct capitalization (eg sImon -> Simon)
  • Remove unwanted binary characters
  • Remove blanks from the start, end and remove multiple whitespace
  • Remove HTML
  • Add line numbers, left and right margins, headers and footers
  • Add or remove columns of text or extra lines
  • Pad fields out to width, or truncate them to a width
  • Extract email addresses
  • and much, much more.

DataPipe runs under Windows 95/98/ME/NT/2000/XP.

See also


What is DataPipe?

Introducing DataPipe

DataPipe is a swiss-army knife for performing text operations on data in live 24x7 production databases where the system cannot be taken down for maintenance and the operation is too complex to perform using vanilla SQL. Operations include:

  • search and replacing with pattern matching, sounds-like or fuzzy matching (multiple search and replace operations in a single pass)
  • fixing the capitalization of text (very useful for names and surnames)
  • Running VBScript or JScript (or PerlScript, PythonScript, REXXscript etc) macros against columns
  • converting between ASCII and EBCDIC, OEM and ANSI, Unicode and ANSI
  • adding or removing left and right margins, headers, footers and line numbers
  • removing extra whitespace
  • forcing text to a specified width
  • removing blank lines
  • fixing common punctuation mistakes
  • removing garbage characters
    •
  • and much, much more

DataPipe allows all user settings to be saved and opened so that the same tasks can be repeated in the future without re-keying.

See also


What's New in v2.8
==================

*   Added control over the cursor type, cursor location and lock
    type.
*   A warning is written to the log file if the desired cursor/lock
    option is not available from the data provider.
*   Now traps errors like the record being deleted.
*   If errors are detected, the error label can be clicked to show
    the log file.
*   Tip of the Day now defaults to On.

What's New in v2.7
==================

*   Added restart control so a job can be resumed at a given record
    if it is cancelled or stopped.
*   Added logging to generate a detailed audit trail.
*   A default query is now inserted when the query string is blank.
*   Right clicking on a column can now be used to copy the column's
    data to the clipboard, which can then be easily used to test the
    TextPipe filter.
*   When a filter edit button [...] is clicked to create a new
    filter, DataPipe adds text to TextPipe's comment tab to say
    which column and which query the filter is designed for, and
    also adds sample data to the Trial Run Area tab when sample data
    is available.
*   Status window now stays on top.
*   Improved handling of Asian languages.
*   Now beeps on job completion.

What's New in v2.6
==================

*   Added missing file to distribution - stdvcl40.dll. This caused
    the error "Error loading type library/DLL".
*   Fixed a problem when selecting a recently used file that you
    could be prompted to save changes twice.

What's New in v2.5
==================

*   New 'Edit' menu with Cut, Copy and Paste.
*   Table names list is now only updated after the database
    connection string has changed.
*   Default row height is now saved, and is copied from the data
    preview grid to the sample results grid. This value is also
    saved with the settings file.

What's New in v2.4.1
====================

*   Requires TextPipe Pro 6.2.8 or later for better integration.
*   New menu item and toolbar button for 'New' settings.
*   New menu item to Hide TextPipe Pro
*   Transform Grid now resizes the filter name columns to its
    maximum.
*   Table names are only queried when the data string has been
    completely changed, not during changes. This prevents errors
    popping up for each key press.
*   If changes have been made to the settings, you are prompted to
    save them prior to opening a new file or the settings being
    cleared.
*   The history of previous filter names now drops down 10 rows.
*   The list of transformation filters are now checked by DataPipe
    (instead of TextPipe - to see if they exist before a preview or
    main run is initiated. This makes errors quicker to identify and
    handle.

What's New in v2.4
==================

*   New demo database and sample modifications are included in the
    installation.
*   Better integration with TextPipe Pro (now requires TextPipe Pro
    6.2.7 or later).
*   New Tip of the Day dialog.
*   Grids now show update messages when preview data can't be
    displayed.
*   Go button now updates immediately when it is pressed to give
    visual feedback.
*   The process of starting a job is now much faster, as pre-
    validation as combined with another stage.
*   Errors are now displayed if field values are incorrect e.g. if
    an empty value is assigned to a NOT NULL field, or if a multi-
    line value is assigned to a single line field etc.

What's New in v2.3.2
====================

*   New status messages to indicate what is being processed.
*   'Save' and 'Save As' now add the filename to the recent files
    list.
*   The initial folder for the Save and Open dialogs now defaults to
    the folder of the last loaded file.
*   The preview grid now copies its column widths from the sample
    data grid.
*   The grids now update immediately when the scroll bar is moved,
    rather than after it is released.
*   Far better handling of error conditions when filters cannot be
    accessed.

What's New in v2.3.1
====================

*   Added 'Reconnect to TextPipe' menu item in case of RPC server
    errors.
*   Fixed display bug with preview grid.

What's New in v2.3
==================

*   Improved handling of NULL values. Now these are passed to and
    from TextPipe Pro as the text value '<NULL>'.
*   Refresh button now refreshes either the data preview or the
    results preview, depending on which tab sheet is currently
    displayed.
*   Much faster - now uses early binding interface.
*   New options dialog to set the display language.
*   Russian translation available.
*   Fixed bug with handling of table names with embedded spaces -
    DataPipe now quotes these.

What's New in v2.2
==================

*   Improved interface to TextPipe Pro makes it easy to use and edit
    filters
*   User Interface can now be translated to other languages

What's New in v2.1
==================

*   Initial release!


Installation

VERY IMPORTANT! BOTH TextPipe Pro AND DataPipe MUST be installed on the SAME computer.

DataPipe comes with TextPipe Pro - and TextPipe MUST be installed on the same machine as DataPipe. For installation, please follow these steos:

  1. Install TextPipe Pro
  2. Install DataPipe.

DataPipe is simple to install - just double-click datapipe.exe that you downloaded, and similarly for textpipepro.exe.

The installation program creates uninstall entries under Control Panel:Add/Remove Programs. These allow DataPipe and TextPipe Pro to be cleanly removed from your system.

Uninstallation

Uninstallation is completely automatic.

To uninstall DataPipe, click the Start Menu, Settings, Control Panel, Add/Remove Programs. Select DataPipe from the list presented, and click Add/Remove. DataPipe will then be deleted. A similar process can be followed for TextPipe Pro.

The uninstall does not remove files modified or added by you, so any changes you have made to files will NOT be lost.


Online Web Purchase

The commercial version of DataPipe can be purchased and received immediately on the Internet, eliminating shipping and handling costs. The latest price list can be downloaded from http://www.crystalsoftware.com.au/order.txt

Direct Purchase URL: http://www.crystalsoftware.com.au/buy.html

See also


CRYSTAL SOFTWARE ORDER FORM
===========================
Prices current as at Wednesday, 25 September 2002
If you are using NotePad, please maximize the window before
printing.

All our products come with (unless stated below)
     * 60 days FREE email support
     * 60 days FREE updates
     * Access to user support groups
     * Notification of new releases, updates and upgrades
     * Access to pre-release products. You will receive access to the
         newest tools created by Crystal Software before they are
         released to the public.

You can also take advantage of our Premium Support Plans (see below).

I N T E R N E T   O R D E R S
-----------------------------

Please go to

     http://www.crystalsoftware.com.au/buy

to pay by credit card using the Internet.

P H O N E   O R D E R S
-----------------------

You can order by credit card over the phone if you do not have
Internet access.

     USA Office: +1 724-850-8186
     USA and Canadian customers: call 1-800-903-4152
     German Office: +49-221-31088-20

Please have the Product Id (see the Product list below) and
your credit card ready.

F A X / M A I L   O R D E R S
-----------------------------

You can order by fax or postal mail if you do not have Internet
access. Please use the order form below for:

     * Purchase orders (commercial, government, educational etc)
     * Fax orders
     * Postal mail orders
     * Cash payments
     * Check payments
     * All other non-internet orders

1. Please download the latest version (*) of this form from

     http://www.crystalsoftware.com.au/order.html

     (*) Prices may not be honored if you do not use the
     latest version

2. Print out and complete this form

3. Fax or mail it to:

USA Office                   German Office

ShareIt! Inc.                ShareIt!
PO Box 844                   element 5 AG
Greensburg                   Vogelsanger Str. 78
PA 15601-0844                50823 Köln/Cologne
U.S.A.                       Germany

Tel. +1-724-850-8186         Tel: +49-221-31088-20
Fax. +1-724-850-8187         Fax: +49-221-31088-29

C O N T A C T   D E T A I L S
-----------------------------

 First Name: _________________________________________________

  Last Name: _________________________________________________

     E-mail: _________________________________________________

     ** Please do not forget to include your e-mail address **
        *** We will use e-mail to communicate with you. ***

    Company: _________________________________________________

    Address: _________________________________________________

             _________________________________________________

             _________________________________________________

             City: ___________________________________________

             State: __________________________________________

             Postal code: ____________________________________

             Country: ________________________________________

      Phone: _________________________________________________

        Fax: _________________________________________________

How would you like to receive the registration key/full version?
(Please tick one)

     [ ] E-mail (please ensure you have filled it in above)
     [ ] Fax (please ensure you have filled it in above)
     [ ] Postal mail (additional charge may be levied)

P A Y M E N T
-------------

How would you like to pay the registration fee? (Please tick
one)

     [ ] Credit card (please fill out the details below)
     [ ] Wire transfer
     [ ] EuroCheque
     [ ] Cash

Credit card information (if applicable)
     [ ] Visa
     [ ] Eurocard/Mastercard
     [ ] American Express
     [ ] Diners Club

Card holder: ______________________________________________

Card Number: ____________________________ Expiry Date: ________

Date / Signature: _________________________________________

P R O D U C T S
---------------

Please note: All prices are in $US. If you wish to order in Euros,
please indicate this when you order and will perform the appropriate
conversion.

** Please contact us for other licensing needs **

ID     DESCRIPTION AND PRICE

Premium Support Plans
143504 __ x 1 year email support for single user
       license
              (Includes updates)
              @  $50 per single user license: $______

146467 __ x 1 year email support for site/server
       license
              (Includes updates)
                                    @  $300 : $______

TextPipe Pro (fix text fast in web sites, database
extracts and e-text up to 2GB!)
       Note: More features than Standard version.
       Please ensure you are evaluating the Pro
       version before ordering

131686 __ x Single User License     @   $199: $______
131820 __ x 5 User Pack             @   $820: $______
131821 __ x 10 User Pack            @  $1850: $______
131822 __ x Server License          @   $599: $______
136051 __ x Site License            @  $4990: $______
136049 __ x Enterprise License      @ $14500: $______

TextPipe Standard
       Note: Less features than Pro version. Please
       ensure you are evaluating the Standard version
       before ordering

137374 __ x Single User License     @    $99: $______

TextPipe Web
       Note: Less features than Standard and Pro
       versions. Please ensure you are evaluating the
       Web version before ordering

157097 __ x Single User License     @    $40: $______

TextPipe Pro Run Time
       Note: (the run time, read-only version of
       TextPipe Pro - filters CANNOT be modified by
       clients)

147437 __ x Single User License     @   $149: $______
147438 __ x Server License          @   $249: $______

TextPipe Engine Standard (standalone .dll)
       Starts from $500 one-off fee (no royalty) for
       inclusion of TextPipe Engine Standard in your
       application.

       Please contact us for a full price schedule

132910 __ x Engine Standard (per app) @ $500: $______

ResumePipe Pro (convert emails and attachments to text
files) - includes TextPipe Pro filtering
137750 __ x Starter pack (1 user)    @ $3995: $______
139756 __ x Additional user licenses @ $2500: $______
139425 __ x Yearly maintenance       @   15%: $______
            (@ 15% of the total purchase price,
             entitles the buyer to all upgrades
             and support for one year)

ResumePipe Lite (convert emails and attachments to text
files) - does NOT include TextPipe Pro
151487 __ x Starter pack (1 user)    @ $2300: $______
151488 __ x Additional user licenses @ $1500: $______
139425 __ x Yearly maintenance       @   15%: $______
            (@ 15% of the total purchase price,
             entitles the buyer to all upgrades
             and support for one year)

DataPipe (interactive search and replace in databases)
138135 __ x Single User License     @   $590: $______
140629 __ x Server License          @  $1390: $______
155969 __ x Site License            @  $6900: $______
139425 __ x Yearly maintenance      @    15%: $______
            (@ 15% of the total purchase price,
             entitles the buyer to all upgrades
             and support for one year)

WordPipe (search and replace in multiple Word
documents)
138134 __ x Single User License     @   $129: $______
138332 __ x 5 User Pack             @   $600: $______
138333 __ x 10 User Pack            @  $1150: $______
138334 __ x Site License            @  $2000: $______

DetachPipe (save, detach and delete email attachments)
146668 __ x Single User License     @   $129: $______
166349 __ x Server License          @   $699: $______
151810 __ x Enterprise License      @  $2000: $______

Time Zone Helper (show international working time
overlap)
162196 __ x Single User License     @    $20: $______

DirSize disk space monitor
132558 __ x Single User License     @    $40: $______
132559 __ x 5 User Pack             @   $180: $______
132560 __ x 10 User Pack            @   $340: $______
140086 __ x Server License          @   $349: $______
140426 __ x Site License            @   $599: $______

DirDate
132564 __ x Single User License     @    $20: $______
132565 __ x 5 User Pack             @    $90: $______
132566 __ x 10 User Pack            @   $170: $______

Arc Menu (formerly Launch)
132561 __ x Single User License     @    $20: $______
132562 __ x 5 User Pack             @    $90: $______
132563 __ x 10 User Pack            @   $170: $______
157148 __ x Enterprise License      @  $2000: $______

ZeroIn
100503 __ x Single User License     @    $20: $______
132049 __ x 5 User Pack             @    $90: $______
132557 __ x 10 User Pack            @   $170: $______

BabyShield
131903 __ x Single User License     @ $19.99: $______

                                     ===================
                                     Total: $US
                                     ===================


Purchase Information

DataPipe is developed as a commercial product, and you may evaluate it for a period of 14 days. You must purchase DataPipe if you use it beyond the evaluation period or if you use any of the output that it generates during the evaluation period. Use of DataPipe after the evaluation period is illegal and violates international copyright laws.

Benefits of Purchasing DataPipe

By purchasing DataPipe you will obtain the following benefits:

  • the latest version of DataPipe
  • access to technical support
  • access to upgrades and bug fixes
  • preferential upgrade pricing
  • notification of future updates
  • you will not be breaking the law
  • you will have a clear conscience
  • you will ensure continued development of this product, making DataPipe an even better product in the future

Purchasing

There are two ways to purchase DataPipe:

  • The easiest, quickest and most convenient way is to visit the DataPipe home page on the Internet at http://www.crystalsoftware.com.au and use the online purchase facility. This facility is for credit card payment only.
  • Mail or email the order form to Crystal Software. The mailing address is contained in the form. The order form is contained in the help file and in ORDER.TXT

See also


Other Documentation

Contact Information
 Other Crystal Software Products
 Shareware Vendor Information
 Distribution Files
 License Information
 Corporate Modifications


Contact Information

For technical support, please see the support page.

For sales, please contact our sales department directly:

Envelope sales@crystalsoftware.com.au

For all other enquiries, please write to our general address:

Envelope mail@crystalsoftware.com.au

Phone/Fax

Phone - phonen1.gif (1029 bytes) +61-3 9888 3104 (9am-5pm Australian Eastern Standard Time - please see below). Within Australia please call (03) 9888 3104
Fax - Fax machine +61-3 8610 1234. Within Australia, please fax (03) 8610 1234.

Time Zones (GMT+10)

Here you can check if you are calling us in Australian business hours (9am - 5pm Monday to Friday)

Your Time | Your Date
Melbourne Time | Melbourne Date

Mail

Crystal Software
2 / 9 Woodbine Court
Wantirna
Victoria 3152
AUSTRALIA

Feedback and Comments

We're always open to feedback and comments! Please use our suggestion box to let us know if there is a feature you would like to see in future versions. We also welcome any reports of bugs or incompatibilities, inconsistencies, spelling mistakes, grammatical errors, broken links etc.


Other Crystal Software Products

Crystal Software develops system utilities for power users, administrators and programmers. It has developed 14 software titles since 1990:

More information about TextPipe TextPipe | Automated search and replace and text processing tool
More information about DataPipe DataPipe | Uses TextPipe Pro to modify database records live in your databases
More information about ResumePipe ResumePipe | Extract emails and their attachments from Outlook, Lotus Notes, Applix or MAPI-compliant email systems to sequentially numbered files, and post-process using TextPipe Pro
More information about WordPipe DetachPipe | Save, detach and delete email attachments from Outlook, Lotus Notes, Applix or MAPI-compliant email systems
More information about WordPipe WordPipe | Search and replace in MS-Word documents using Word-like search/replace features or TextPipe Pro
More information about Time Zone Helper Time Zone Helper | Displays the working time overlap for people in different time zones
More information about DirSize DirSize | Windows 95 command line utility to accurately show used and wasted disk space, sorted by total size, depth, wastage etc
More information about ZeroIn ZeroIn | Fast CD replacement with spelling correction, history, aliases and long filename support
More information about Arc Arc Menu | Text-based, mouse-driven windowed menu system with many advanced capabilities
More information about DirDate DirDate | Unique ability to change the date on folders
More information about BabyShield BabyShield | Temporarily disables the mouse AND keyboard while your pet or child taps on the keyboard
More information about ClipSize ClipSize | Displays memory occupied by the clipboard and allows it to be cleared easily
More information about Clean 'n' Go Clean 'n' Go | Temporarily disables the mouse or keyboard while they are cleaned.
More information about LFNit! LFNit! Restores Windows 95 long filenames. Designed to allow continued use of old 16 bit applications (eg XTree, tape backup software, LapLink, PkZip etc) without losing long filenames

Contact Information


Vendor Information

If you are a

  • Shareware distributor
  • Disk vendor
  • Web site
  • CDROM publisher
  • Magazine
  • or any other organisation or person

you may distribute the Evaluation Editions (*) of all Crystal Software programs provided the following conditions are met:

  1. You must inform the buyer that they are paying for your service, not for the programs, and that the programs must be purchased if they wish to continue using them after the evaluation period.
  2. You must include all original files included with the program, and you must not modify these files in any way. If required, you may add new files of your own.
  3. You must let us know by email if you include our software in your collection. We can then keep you informed of the latest version.

(*) All programs use the Help\About dialog or the -? Command line parameter to identify that they are an Evaluation Edition.

Contact Information


Distribution Files

The DataPipe archive consists of the following files:

DATAPIPE.EXE | The DataPipe program
STDVCL40.DLL | Required DLL (this was missing from earlier versions, and caused "Error loading type library/DLL" errors.
*.DPP | Useful filters for common tasks
DESC.SDI | Archive description file
FILE_ID.DIZ | Archive description file
ORDER.TXT | Order form
DATAPIPE.CHM | Help file for 32-bit version
VENDOR.TXT | Vendor information
CORPORAT.TXT | Information about Corporate versions
LICENSE.TXT | User license information
PRODUCTS.TXT | Information about other Crystal Software products
WHATSNEW.TXT | What's new in this version of DataPipe

If any of these files are missing from your copy of DataPipe then you can obtain a complete version from the Crystal Software Home Page on the Internet.


    This program and the accompanying files and documentation are

              Copyright (c) 1990- 2002 Crystal Software
                         All rights reserved

This file contains our end user license.  It also explains the terms
of the Commercial License where they're different from the Evaluation
License.

If you have any questions, please contact us at

     Address: Crystal Software
              2 / 9 Woodbine Court
              Wantirna
              Victoria 3152
              AUSTRALIA
       Phone: (+61) 3 9888-3104 (GMT+10)
         Fax: (+61) 3 8610-1234
       Email: sales@crystalsoftware.com.au
         Web: http://www.crystalsoftware.com.au

====================================================================
Plain English Evaluation License Agreement and Limited Warranty

1. SUMMARY.  You may install this SOFTWARE on as many computers as
your license allows, and no more:

     You may access the software through a network, provided that you
     have obtained individual licenses covering all computers that
     will run the software from the network. For instance, if six
     different workstations will access the software on the network,
     you will need a six-user license, whether they use the software
     at different times or concurrently.

     * Single User License/Desktop Edition/Personal Edition/Single
     User Pack
     May either be used by a single person who uses the software
     himself on one or more non-server class computers, or installed
     on a single non-server class computer and used by multiple
     people, but not both.

     * 5 User Pack
     May be installed on UP TO FIVE (5) non-server class computers
     ONLY

     * 10 User Pack
     May be installed on UP TO TEN (10) non-server class computers
     ONLY

     * Server License
     May be installed on ONE (1) server class computer ONLY

     * Site License/Educational Site License
     May be installed on UP TO 500 computers of server class or non-
     server class at one site ONLY

     * Enterprise License/Company Edition/University Edition
     May be installed on UP TO 2000 computers of server class or non-
     server class  directly owned by a Company or University ONLY
     (but not on client's computers)

This is a legal agreement between you and CRYSTAL.  By using the
SOFTWARE you are agreeing to be bound by the terms of this Agreement.
Do not use the SOFTWARE if you do not agree to the terms of this
Agreement.

2. DEFINITIONS.  In this agreement:

     *    "SOFTWARE" means the computer program you have purchased
          or obtained for evaluation, including the program,
          associated documentation and other materials;

     *    "Evaluation Copy" means a copy of the SOFTWARE distributed
          by us or by our authorised agents for evaluation purposes
          and which is described as an "Evaluation Edition" or a
          "Shareware Edition" by the SOFTWARE's "About" screen;

     *    "Commercial Copy" means a copy of the SOFTWARE purchased
          from us or from a dealer, and is NOT described an an
          Evaluation Copy by the SOFTWARE's "About" screen.  The
          actual description may include "Desktop Edition", "Single
          User Edition", "Personal Edition", "Server Edition", "Site
          Edition" or "Company Edition" or other variations
          depending on the version purchased;

     *    "you" means the end user of the SOFTWARE.  This can be
          either an individual or an entity;

     *    "server class" means a computer designated as a file
          server, database server, application server or other kind
          of server;
          "desktop class" or "non-server class" means a computer
          where only one person uses it at a time;

     *    "CRYSTAL" means Crystal Software, an Australian
          corporation (address shown above).

3. GRANT OF LICENSE.  CRYSTAL grants you the right to INSTALL this
software on as many computers as you have purchased a license for.
You may NOT install it on computers that you do not have a license
for.

The SOFTWARE is "installed" on the computer when it is loaded into
permanent memory (e.g. hard disk, CD-ROM, or other storage device) of
the computer.

4. COPYRIGHT.  The SOFTWARE is owned by CRYSTAL and is protected by
copyright laws and international treaty provisions and all other
applicable national laws.  Therefore, you must treat the SOFTWARE
like any other copyrighted material (e.g. a book or musical
recording) except that you may make one copy of the SOFTWARE solely
for backup or archival purposes.  You may not copy the user
documentation provided with the SOFTWARE.

5. OTHER RESTRICTIONS.  You may not rent, lease, sell, or attempt to
rent, lease or sell the SOFTWARE. You may transfer the SOFTWARE and
accompanying written materials on a permanent basis provided you
retain no copies and the recipient agrees to the terms of this
Agreement.

You must not copy, translate, adapt, modify, alter, decompile,
disassemble, or otherwise reverse engineer or create any derivative
work of the SOFTWARE, merge the SOFTWARE or any part thereof with any
other software or change any SOFTWARE in whole or in part, nor allow
any other party to do so.  You must not alter or remove any copyright
or other intellectual property notifications applied to the SOFTWARE.

You acknowledge that the SOFTWARE and the copyright, and other
intellectual property rights of whatever nature in the SOFTWARE are
not your property and contain valuable proprietary information and
trade secrets and that this license confers no title or ownership in
the SOFTWARE and no rights in any associated source code, and is not
a sale of any rights in SOFTWARE.

6. EVALUATION PERIOD.  You may use an Evaluation Copy of the SOFTWARE
for up to 14 days in order to determine whether it meets your needs.
While you are evaluating the SOFTWARE, you may use it on as many
computers as are required to perform your evaluation.  Your
evaluation period begins when you first install the SOFTWARE on one
or more computers for evaluation purposes.  Once the evaluation
period is over you MUST PURCHASE THE SOFTWARE or REMOVE IT FROM ALL
COMPUTERS. If you use the software for commercial purposes DURING
your evaluation you MUST PURCHASE THE SOFTWARE.

7. MAKING AND DISTRIBUTING EVALUATION COPIES OF THE SOFTWARE.  You
may make copies of your Evaluation Copy of the SOFTWARE to give to
others, as long as you include all of the files that you originally
received with your Evaluation Copy.  When you give an Evaluation Copy
of the SOFTWARE to another person, you agree to inform them that
their copy is to be used for a time-limited evaluation period, and
that they must purchase a Commercial Copy if they continue to use the
SOFTWARE once the evaluation period has ended.  You agree not to sell
Evaluation Copies of the SOFTWARE or distribute them to others for
any kind of compensation or fee unless the fee solely covers your
costs in providing it to the end user.

8. OPERATION OF THE SOFTWARE.  The SOFTWARE offers powerful
capabilities.  If you do not follow our instructions, or if you use
the SOFTWARE improperly, you can destroy files or cause other damage
to your software and data.  You assume full responsibility for the
selection and use of the SOFTWARE to achieve your intended results.

9. LIMITED WARRANTY.  The following limited warranty applies to
Commercial Copies of the SOFTWARE.  It is included here so you
understand what your warranty will be when you purchase a Commercial
Copy. This warranty does NOT apply to Evaluation Copies of the
SOFTWARE.

Bugs and SOFTWARE Errors. We don't promise that the SOFTWARE will be
free of bugs or errors. If you report an error or bug to us, we will
use our best efforts to correct it.

10. DISCLAIMERS AND LIMITATIONS.  And now the legal language we
couldn't avoid:

DISCLAIMER FOR EVALUATION COPIES: YOU ARE NOT PAYING FOR THE TRIAL
USE LICENSE FOR YOUR EVALUATION COPY OF THE SOFTWARE.  ACCORDINGLY
THE EVALUATION COPY IS PROVIDED "AS-IS" AND WITHOUT WARRANTY OF ANY
KIND.

DISCLAIMER FOR COMMERCIAL COPIES: Except for the limited warranty
stated above, we disclaim any and all other warranties, express or
implied, oral or written, including any implied warranties of
merchantability or fitness for a particular purpose.  The limited
warranty above gives you specific legal rights, but you may have
other rights, depending on where you purchased the SOFTWARE.

LIMITATION OF LIABILITY.  IN NO EVENT SHALL WE BE LIABLE FOR ANY
INCIDENTAL, CONSEQUENTIAL, OR PUNITIVE DAMAGES WHATSOEVER ARISING OUT
OF USE OF THE SOFTWARE OR YOUR RELATIONSHIP WITH US.  SOME COUNTRIES
OR STATES DO NOT ALLOW EXCLUSION OR LIMITATION OF LIABILITY FOR
INCIDENTAL OR CONSEQUENTIAL DAMAGES, THEREFORE THE ABOVE LIMITATION
MAY NOT APPLY TO YOU.

11. This agreement is to be interpreted under the laws of the state
of Victoria, Australia.

END OF LICENSE AGREEMENT


Corporate Modifications

Corporate and business users are welcome to approach Crystal Software for modifications specific to their needs. We can easily incorporate any changes you desire. Please send us specifications and we will send you a quote. Please also include details such as a contact name, phone and/or address, plus any other details you think are relevant.

Contact Information


File Menu

Open

Opens settings from a saved DataPipe settings file.

Save

Saves the current settings to a DataPipe settings file.

Save As

Saves the current settings to a DataPipe settings file with a new name.

Recent files

This submenu displays a list of any recently opened DataPipe settings files. Click the menu item to open it.

Exit

Closes DataPipe.

See also


File Menu

Edit Menu

Cut

Cuts the selected text to the clipboard.

Copy

Copies the selected text to the clipboard.

Paste

Pastes text from the clipboard.

See also


Options Menu

Logging, Restart and Commit

This menu opens the Logging, Restart and Commit options dialog.

Change language

This menu item allows you to change the display language used by DataPipe. To translate DataPipe into your language, or to find what translations are available, please visit http://www.crystalsoftware.com.au/languages.html

See also


Tools Menu

Refresh

If the current tab is the Query or Sample Data tab, this menu item refreshed the Sample Data using the query or table name specified on the Query tab. If the current tab is the Transforms or Preview Changes tab, the results preview data is refreshed.

Show TextPipe Pro

This forces the TextPipe Pro window to become visible. The TextPipe Pro window is normally minimized.

Hide TextPipe Pro

This forces the TextPipe Pro window to minimize.

Reconnect to TextPipe Pro

If an unexpected errors occurs, this menu item will force DataPipe to connect to a new copy of TextPipe. The old copy of TextPipe will remain running until you force it to close manually, either using Ctrl+Alt+Del, selecting the process and choosing 'End Process', or click 'Force Close' in TextPipe's task bar right click menu.

See also


Help Menu

Contents

Displays the contents page of the DataPipe online help.

Index

Displays the index page of the DataPipe online help.

Show Tip of the Day

Shows the tip of the day dialog

Buy DataPipe Now!

Opens your web browser at http://www.crystalsoftware.com.au/buy.html to purchase DataPipe.

Order form

Opens a text-based order form to order DataPipe.

DataPipe Home Page

Opens your web browser at http://www.crystalsoftware.com.au/datapipe.html

Crystal Software Home Page

Opens your web browser at http://www.crystalsoftware.com.au

Check web for DataPipe updates

Uses the internet to find out if a new version of DataPipe is available. If so, you can choose to go to the Crystal Software web site to download the update.

Subscribe to DataPipe update list

Join the DataPipe email list to receive notification of program updates and new releases. To join, send an email to

datapipe-subscribe@yahoogroups.com

or go the following web page.

Technical support

Opens your web browser at http://www.crystalsoftware.com.au/support.html

About

Displays DataPipe's About window.

See also


Query

Data source

The Data source provides DataPipe with a database connection string that gives the name of the database and how to connect to it. We recommend using the Build button unless you are familiar with setting the connection string manually.

How do I connect to a MS Access 2000 database?
Provider=Microsoft.Jet.OLEDB.4.0;DataSource=C:\MyDatabase.mdb;Persist Security Info=False

How do I connect to a password protected MS Access 2000 database?
Provider=Microsoft.Jet.OLEDB.4.0;Jet OLEDB:Database Password=XXXXXX;DataSource=C:\MyDatabase.mdb;Persist Security Info=False

What provider should I use for MS Access
For MS Access 97 use Microsoft.Jet.OLEDB.3.51
For MS Access 2000 use Microsoft.Jet.OLEDB.4.0

How do I connect to a dBase database?
Provider=Microsoft.Jet.OLEDB.4.0;DataSource=C:\MyDatabase.mdb;Extended Properties="dBase 5.0;"

How do I connect to a Paradox database?
Provider=Microsoft.Jet.OLEDB.4.0;DataSource=C:\MyDatabase.mdb;Extended Properties="Paradox 7.X;"

Build

The Build button allows the connection information to be constructed. A username and password can be specified in the database connection string if required. A file containing the connection information can also be specified.

SQL query string

The SQL query string is a SQL command that tells DataPipe which rows of data to retrieve. The SQL command must be a SELECT statement - if it is not the dataset returned may not be updateable (it cannot be modified). DataPipe will warn you if the dataset is not updateable when you click the Preview data button. Click the Parse query button to check if the syntax of your SQL query string is correct.

To ensure an updateable record set, please ensure that you select ALL fields of the table e.g. your query should start with 'select * from .....'. This ensures that the database driver has enough information (and all the key fields) to be able to update the original record once it has been modified.

Avoid returning too many fields. Performance suffers as a larger number of fields are returned. For example, avoid using "SELECT * FROM TableName" when TableName has 40 fields, and you really need to modify only 2 or 3 fields. Only SELECT the fields you need to modify.

How do I use multiword table / field names (spaces in Table or Field name)?
Enclose multiword names in [ ] brackets:
SELECT [Last Name], [First Name] FROM [Address Book]

How do I use constant fields in an SQL query?
SELECT ''2002'', [First Name], Salary FROM Employees

How do I limit the records returned in an SQL query?
SELECT TOP 100 [First Name], Salary FROM Employees

You can also limit the number of records displayed by using the Rows to Display field on the Sample data tab.

Preview data

The Preview data button uses the SQL query string to retrieve a group of sample records from the Data source. The data is then displayed on the Sample data tab.

Parse query

Click the Parse query button to check if the syntax of your SQL query string is correct.

See Also

< Previous | Query | Sample data | Transforms | Preview changes | Go | Next >


Logging, Restart and Commit

Logging, Restart and Cursor options

This dialog controls logging, restart and cursor options.

Logging

No logging

Errors are not logged to a file. This is the default.

Log actions to file

Actions such as job start and user name, job end, restart parameters, the query being used, and any erroneous records are all appended to the log file. A typical log file looks like this:

DataPipe Log File

date,time,level,message
2002-09-11,13:37:00,Info,Log file started
2002-09-11,13:37:00,Info,Job started by [Simon Carter]
2002-09-11,13:45:21,Info,Restarting at record 2
2002-09-11,13:37:06,Error,Invalid column value set on row 10, column "price"
  Old Value: <NULL>
  New Value: AAAAAAAAAAAA
  Error: Type mismatch
2002-09-11,13:43:06,Info,Complete. 70 rows processed
2002-09-11,13:43:06,Info,End of job

Restart

No restart

All records are processed. This is the default.

Restart from record

DataPipe starts changes from the specified record. This can be used to restart a job that you cancelled.

Cursor and Lock options

Note: DataPipe always moves forward through a result set, so it does not require options that provide bi-directional scrolling.

Cursor location

Determines whether to use a client-side or server-side cursor library. DataPipe's default is Use Server.

A client-side cursor offers more flexibility. All data is retrieved to the local machine and then operated on there, allowing operations not normally supported by servers like sorting and resorting the data and additional filtering. SQL statements are executed at the server, so for statements that restrict the result set with a WHERE clause, only the reduced result set is retrieved to a local cursor.

A server-side cursor offers less flexibility, but may be more advantageous (or necessary) for large result sets. Using a server-side cursor becomes necessary when the sheer size of a result set exceeds the available disk space that would be needed to create the client-side cursor. Also, many servers only support unidirectional cursors. This would preclude moving the record pointer in the dataset backward (even one record) through the result set.

Cursor type

Indicates the type of cursor used for the record set when it is opened. Cursor type affects directional scrolling through a record set and the visibility of changes made by other users. DataPipe's default is Dynamic.

Cursor Type | Meaning
Unspecified | A specific cursor type has not yet been specified.
Forward Only | Forward-only cursor in which you can only scroll forward; better performance because only requires single through record set.
Keyset | Keyset cursor; records added by other users are not visible; records deleted by other users are inaccessible.
Dynamic | Dynamic cursor; data modifications, deletions, and insertions by other users are visible; scrolling forward and back allowed.
Static | Static cursor; static copy of records; data changes by other users not visible; used mostly for reporting.

Note: Only Static is supported if the Cursor Location is set to Use Client.

Note: If the requested cursor type is not supported by the provider, the provider may return another cursor type.

Lock type

Specifies the type of record lock used during editing. DataPipe's default is Optimistic.

Lock Type | Meaning
Unspecified | A specific lock type has not been specified or determined. The provider will use the default locking scheme.
Read Only | Read-only; the current execution cannot modify data.
Pessimistic | Pessimistic locks; locks instantiated on a record-by-record basis when edit mode is applied to a record.
Optimistic | Optimistic locks; locked are initiated on a record-by-record basis, but only when updates are applied and a record is being updated.
Batch Optimistic | Optimistic batch updates; used for batch update mode rather than immediate update mode.

Note: To use a specific locking scheme, the provider and database type must support that locking scheme. Use Unspecified to have the provider use the default locking scheme. Use the Lock Type Batch Optimistic to open a record set in batch update mode.


Sample data

The Sample data tab shows data retrieved from the Data source using the SQL query string. The amount of data displayed can be controlled using the Rows to display field.

Rows to display

Use the Rows to display field to limit how many rows are returned as a preview of the data source. These rows of data will also be used to preview the changes on the Preview Changes tab. All available records (displayed or not) will be changed when the Go button is clicked to process the entire set of records. If you want to limit the actual rows of data returned, you can try using a 'SELECT TOP 100 * FROM <TABLENAME>' type statement (depending on your database).

Refresh button (on toolbar)

Press F5 or click the refresh button on the toolbar to refresh the preview after you have modified the transform on the Transform tab.

See Also

< Previous | Query | Sample data | Transforms | Preview changes | Go | Next >


Transforms

The transform tab is where you specify how to change each column of data returned by the SQL query string.

Column name

This is a read-only column that displays the column names in the data set returned by the SQL query string. If the query or table name changes, TextPipe will retain the filter settings for column names that remain the same, even if their order changes.

TextPipe Pro Filter Filename

This column specifies the TextPipe Pro filter to use to modify the data in this column. You can manually enter a filename (.fll file) or you can choose a recently used filename from the drop-down list. The Input filter and final Output filter of the TextPipe filter are ignored - the text comes from the given column and the modified result is returned to the specified column. For more details, please see handling of null values.

Browse

Clicking the button in this column opens a browse dialog for you to locate a TextPipe filter to use.

Edit

Clicking the button in this column edits the filter shown in the same row, or if none is specified, opens TextPipe so that a new filter can be designed.

Preview

Clicking the Preview button displays transformed data from the Sample data tab on the Preview changes tab.

Clear

Clears the filter filename from the current row, or from all selected rows.

Copy to all

Copies the selected row's filter filename to all other rows.

See Also

< Previous | Query | Sample data | Transforms | Preview changes | Go | Next >


Handling of Nulls

All of TextPipe Pro's processing is done with pure text - TextPipe has no understanding of 'NULL' data values. When DataPipe encounters a NULL column value, it converts it to the text value '<NULL>'. You will see this value in the data preview tab and in the result preview tab.

Similarly, if TextPipe Pro passes back a text value of '<NULL>', DataPipe converts it to the data value of NULL and copies it into the column.

If you wish, you can prevent TextPipe Pro from processing NULLs using one of the following methods:

  1. Make sure your SQL query excludes NULL values e.g.

    select * from customers where phone_number is not NULL

  2. Use a 'Select lines non-matching' filter, with a pattern of '<NULL>', and then include your search and replaces etc as sub filters of the Select lines filter.
  3. You can set a column to NULL by using a 'Delete All' filter followed by a Add Header filter with text of '<NULL>'.


Preview changes

The Preview changes tab shows you the affect that the transforms you have specified will have on the sample data currently shown on the Sample data tab.

Refresh button (on toolbar)

Press F5 or click the refresh button on the toolbar to refresh the preview after you have modified the transform on the Transform tab.

See Also

< Previous | Query | Sample data | Transforms | Preview changes | Go | Next >


Go!

The Go tab signals that DataPipe has all the information it needs to begin modifying text live in your database. Click the Go button when you are ready to begin running the transform.

See Also

< Previous | Query | Query data | Transforms | Preview changes | Go | Next >


How can I use TextPipe?

How can I use DataPipe?

Turn to DataPipe whenever the SQL is just too complex, or when you can't even think how to begin to fix up your data.

Use it when you have only 5 minutes before you want to head for home, and you need to run a search/replace against 3 TB of data. A quick test using DataPipe, click the Go! button...and off you go.

See also